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The magnetic properties and Mott transition of the Hubbard model on the square lattice with 
frustration are studied at half-filling and zero temperature by the variational cluster approximation. 
When the on-site repulsion U is large, magnetically disordered state is realized in highly frustrated 
region between the Neel and coUinear phases, and no imcommensurate magnetic states are found 
there. As for the Mott transition, in addition to the Mott gap and double occupancy, which clarify 
the nature of the transition, the structure of the self-energy in the spectral representation is studied 
in detail below and above the Mott transition point. The spectral structure of the self-energy is 
almost featureless in the metallic phase, but clear single dispersion, leading to the Mott gap, appears 
in the Mott insulator phase. 

PACS numbers: 71.30.-|-li, 71.10.Fd, 71.27.+S, 
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I. INTRODUCTION 

Geometric frustration is one of the most impor- 
tant issues in strongly correlated electron systems. 
In addition to the experimental discovery of the 
high Tc cupratesfi for example, the triangular-lattice 
organic materials k-(BEDT-TTF)2X2i"-^ and herbert- 
smithite ZnCu3(OH)6Cl2^— with kagome lattice struc- 
ture have inspired a lot of theoretical interest. In fact as 
a theoretical model incorporating the strong electron cor- 
relations and geometric frustrations, the Hubbard model 
on the square, triangle, and kagome lattices is actively 
studied using non-perturbative methods^r^^i aiming to 
shed lights on the role of these effects on magnetism, 
Mott physics, and superconductivity. 

In this paper, we study the magnetic properties and 
Mott transition in the Hubbard model on the square lat- 
tice with nearest- and next-nearest-neighbor hoppings t 
and t' , and on-site Coulomb repulsion U (see Fig. [T]) at 
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FIG. 1: (a) Lattice structure of the Hubbard model on the 
square lattice with nearest- and next-nearest-neighbor hop- 
pings t and t' . (b) The first Brillouin zone of the square 
lattice (solid outer square). The magnetic Brillouin zones 
corresponding to the AF, AF2, and AFC orderings in Fig. [2] 
are also depicted with dash-dotted (AF), dotted (AF2), and 
dashed (AFC) lines, respectively. 



(a) 
I 
t 

t 

t 



AF 



(b) 
\ 
\ 
t 
t 
\ 
\ 



AF2 



(0 
I 
I 
I 



AFC 
t \ 



w 

t \ 
t \ 



FIG. 2: The spin configurations of the (a) AF (Neel) , (b) 
AF2, and (c) AFC (coUinear) states. 

half-filling and zero temperature using the variational 
cluster approximation (VCA))^"— which is formulated 
based on a rigorous variational principle and exactly 
takes into account the short-range correlations. 

The ordinary Neel state with the ordering wave vector 
Q = (tt, tt), the so-called coUinear phase with Q = (tt, 0), 
and a third state with longer periodicity in real space cor- 
responding to Q = (tt, 7r/2) are considered for the mag- 
netic orderings and 12-site clusters of the three different 
shapes are used to see the finite-size effects of the calcula- 
tions. Hereafter, the magnetic orderings with Q = (tt, tt), 
(tt, 7r/2), and (tt, 0) are referred to as AF, AF2, and AFC, 
and they are illustrated in Fig. [2j 

We find that, when U is larger than the band width 
W = 8i, the paramagnetic (PM) phase is realized be- 
tween the AF \t'/t < 0.65) and AFC (0.8 < t'/t) phases, 
and there is no area where the AF2 phase is realized. For 
U > W, the cluster-shape dependence of the calculations 
is small and our results agree well with the analysis of the 
Heisenberg modeli^"— in the large U limit. For U < W, 
VGA on the 3x4 cluster shows that there is a small re- 
gion in the phase diagram where the AF2 phase is realized 
(0.7 < t'/t < 0.85 and 4.5 < U/t < 6.5), which qualita- 
tively agrees with the mean-field approximation, though 
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FIG. 3: (Color online) The noninteracting ([/ = 0) Fermi surface at half-filling for (a) i' = 0.2, (b) f' = 0.5, and (c) f' = 0.8. 
The best nesting vector Q is also shown. The change in the topology (Lifshitz transition) takes place around t' ~ 0.71. 



the finite-size eiTects turn out to be non-negligible. 

As for the Mott transition, in addition to the Mott 
gap and double occupancy, which clarify the nature of 
the transition, we study in detail the spectral structure 
of the self-energy in its spectral representation<^M£ We 
found that the Mott transition takes place below the 
band width and is of the second order. The spectral 
structure of the self-energy is almost featureless in the 
metallic phase, but clear single dispersion, leading to the 
Mott gap, appears in the Mott insulator phase. 



II. THE MAGNETIC PROPERTIES IN THE 
MEAN-FIELD APPROXIMATION 

The Hamiltonian of the Hubbard model on the square 
lattice with nearest- and next-nearest-neighbor hoppings 
t and f', and the on-site Coulomb repulsion \J reads 



^ ^ / , ''ij'^ia'^j'^ 



U ^ ni^riii 



^^ 



H'' 



(1) 



where f^ = t if the sites i and j are nearest-neighbor, 
tij = t' if they are next-nearest-neighbor, and tij = 
otherwise, and ^ is the chemical potential. The annihila- 
tion (creation) operator for an electron at site i with spin 

a is denoted as Cja (cj^o-) ^'^'^ "-Jo- = Hcr'^i'y The hopping t' 
introduces frustration to the Neel ordering since it yields 
the antifcrromagnetic interactions between next-nearest- 
neighbor spins, which are parallel in the Neel ordering. 
The energy unit is set as t = 1 hereafter. 

The energy band for the noninteracting case {U — 0) 



£k = —2t yj cos h — At' cos kx cos ky 



(2) 



i=x,y 



(— TT < fci < tt), SO the band width is 14^ = 8i. The 
noninteracting Fermi surface at half-filling is depicted in 
Fig. H for (a) t' = 0.2, (b) t' = 0.5, and (c) t' ^ 0.8 
together with the best nesting vector Q. 



First, we bricfiy study the magnetic properties of this 
model at half-filling in the large U expansion. The lead- 
ing order approximation of this expansion leads to the 
effective spin Hamiltonian ( Ji — J2 Heisenberg model) 



H — Ji 2_^ Si ■ Sj + J 2 2_^ Si ■ Sj 



(3) 



'J 



'J 



where Ji = At'^/U and J2 = At'"^ /U , and the next leading 
corrections to the Hamiltonian ( Bl) a re of the order 1/U^ , 
as are calculated e.g. in Refs. I4ll - t43l The (classical) 
energy per site of the Hamiltonian (jH]) for the magnetic 
orderings with Q = (tt, ir/n) (n = 1, 2, • • • ) is evaluated 
as 

^« = -J{J2 + -(Ji-2J2)}, 
2 n 

where n = 1 for AF, n = 2 for AF2, and n — >■ 00 for 
AFC. Therefore at t' = t[ = t/y/2 ~ 0.707i all the En 
are exactly degenerate in this approximation, and -Eaf < 
Eaf2 < ■■■ < Eafc for t' < t'^ and Eaf > Eaf2 > ■■■ > 

Eafc for i' > ic- 

Next we study the magnetic properties of this model 
in the mean-field approximation taking into account the 
three magnetic states AF, AF2, and AFC. The Hamilto- 
nian in the mean-field approximation is constructed fol- 
lowing the usual procedure and we set the average num- 
ber (riia-) as 

where sign(i) = +1 (—1) for the spin up (down) sites 
in Fig. [21 TT. = 1 (at half-filling), and the parameter A 
becomes the order parameter per site for the consistent 
mean-field solutions. The critical interaction strength Uc, 
above which nontrivial solutions are obtained, is calcu- 
lated at zero temperature as a function of t' in Fig. 0] 
for the AF (circles) , AF2 (triangles) , and AFC (squares) 
orderings. 

The phase diagram is calculated as a function of t' and 
U in Fig.[5]by comparing the energies of the magnetic and 
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FIG. 4: (Color online) The critical interaction strength JJa 
as a function of t' for the AF (circles), AF2 (triangles), and 
AFC (squares) orderings in the mean-field approximation. 
Nontrivial solutions are obtained for U > Uc- Lines are 
guides to the eye. 
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FIG. 5: (Color online) Phase diagram at half-filling and 
zero temperature as a function of t' and U in the mean field 
approximation. The circles, triangles, and squares represent 
the AF, AF2, and AFC states, respectively. In the classical 
approximation of the Ji — J2 Heisenberg model (see text), 
the energies of the AF, AF2, and AFC states are exactly 
degenerate on the vertical dotted line {t' = i^ ~ 0.71) and 
the AF state is stable for t' < t'^ while the AFC state is 
realized for t'^ < t' . 



PM states. At t' — 0.5 the best nesting vector seems to 
be Q = (tt, 0) but according to the mean-field analysis 
the AF state is realized. In the large U region the mean- 
field analysis agrees with the classical approximation of 
the Ji — J2 Heisenberg model, which predicts that the AF 
and AFC phases are separated by the vertical dotted line 
{t' — t'^) in Fig. [SJ Our results are consistent with the 
previous mean-field analyses^^^ studying the AF and 
AFC states. 



III. VARIATIONAL CLUSTER 
APPROXIMATION 

Next we apply VCA3^"— to examine the magnetic 
properties and Mott transitions of this model. VCA is 
an extension of the cluster perturbation theory2^ based 
on the self-energy-functional approachj^l This approach 
uses the rigorous variational principle 



(5f7t[I]]/(5S = 



(4) 



for the thermodynamic grand-potential Ot written in the 
form of a functional of the self-energy E as 



f^tp] = Fm + Trln(-(Go 1 - E)-i), 



(5) 



where F[E,] is the Legendre transform of the Luttinger- 
Ward functional^ and the index t denotes the explicit 
dependence of fit on all the one-body operators in the 
Hamiltonian. The stationary condition (j3]) for i7t[S] 
leads to the Dyson's equation. All Hamiltonians with 
the same interaction part share the same functional form 
of F\T,], and using that property F[E,] can be evalu- 
ated from the exact solution of a simpler Hamiltonian 
H', though the space of the self-energies where F[I]] is 
evaluated is now restricted to that of H'. In VCA, one 
uses for H' a Hamiltonian formed of clusters that are dis- 
connected by removing hopping terms between identical 
clusters that tile the infinite lattice. A possible symmetry 
breaking is investigated by including into H' the corre- 
sponding Weiss field that will be determined by minimiz- 
ing the grand-potential J7f Rewriting i^[E] in Eq. ((S]) in 
terms of the grand-potential il' = il[ [S] and Green func- 



tion G'- 



Gn 



E of the cluster Hamiltonian H' , the 



grand-potential becomes a function of t' expressed as 
iltit') = n'-f l^e^-^lndet (l + {G^' - G'o-')G') , 

JC '^ K 

(6) 
where the functional trace has become an integral over 
the diagonal variables (frequency and super-lattice wave 
vectors) of the logarithm of a determinant over intra- 
cluster indices. The frequency integral is carried along 
the imaginary axis and i5 — >■ -1-0. The stationary so- 
lution of r2t(t') and the exact self-energy of H' at the 
stationary point, denoted as E*, are the approximate 
grand-potential and self-energy of H in VCA, and phys- 
ical quantities, such as expectation values of the one- 
body operators, are calculated using the Green function 
Gq^^ — E*. In VCA, the restriction of the space of the 
self-energies E into that of H' is the only approximation 
involved and the electron correlations within the cluster 
used to set up H' are rigorously taken into account by 
exactly solving H' . 

In our analysis, the three magnetic orderings AF, AF2, 
and AFC are considered as possible symmetry breaking 
patterns, and the 12-site clusters of the three different 
shapes depicted in Fig. [H] (3x4, 2x6, and a cross-shaped 
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FIG. 6: (Color online) The clusters used to investigate the (a) AF, (b) AF2, and (c) AFC states. When necessary, these 
12-site clusters are paired with their rotated or reflected mirror images so that the extended 24-site clusters tile the lattice in 
the Bravais sense in the presence of the Weiss fields. 
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FIG. 7: (Color online) The phase diagram at half-filling and zero temperature as a function of t' and U obtained by 
VGA using the (a) 3x4, (b) 2x6, and (c) 12C cluster. The circles, triangles, and squares represent the AF, AF2, and 
AFC states, respectively. Non-filled marks indicate that their energies are degenerate with other solutions within the ac- 
curacy of 10^''. The crosses are the Mott transition points obtained in each cluster assuming that no magnetic order is allowed. 



12-site cluster, referred to as 12C hereafter) are used to 
set up the cluster Hamiltonian iJ', where the Weiss field 



Hm = hyi} ^ s\gn{i){ni^ - n.^x) 



(7) 



with sign(i) = +1, (—1) for the shaded (white) sites in 
Fig. [5] is included. 

As is shown in Fig. [HI when necessary (3x4 and 12C in 
(a), and 3x4 and 2x6 in (b)), these 12-sitc clusters are 
paired with their appropriate rotated or reflected images 



so that the extended 24-site clusters tile the infinite lat- 
tice in the Bravais sense in the presence of the magnetic 
ordering. In these cases the Green function G" in Eq. (j6]) 
is given by 



G' 



G'- 



G'. 



t 



12 



(8) 



where G"i is the exact Green function on a 12-sitc cluster 
(the site and spin indices suppressed), G'2 is the exact 
Green function of the mirror image (a simple transfor- 
mation of G'l), and ii2 is the hopping matrix linking the 
two 12-site clusters (the sohd and dashed links in Fig. [6]). 
In all cases, the correlations within the 3x4, 2x6, and 
12c clusters are rigorously taken into account by exactly 
diagonalizing H' . The cluster-shape dependence of the 
results is a measure of the finite-size effects in our anal- 
ysis. With this set up, the grand-potential per site Q. 
is now a function J7(/x',/im) of the Weiss field /im and 
the cluster chemical potential /i', where the latter should 
be included for the thermodynamic consistency,— and 
the variational principle (jl|) becomes the stationary point 
search of r2(/x',/iM) with respect to hyi and /i'. During 
the search, the chemical potential of the system jjl is also 
adjusted so that the electron density per site n is equal 
to 1 within the accuracy of 10~* (i.e. |«— 1| < 10""*). In 
general, a stationary solution with hyi 7^ corresponding 
to the magnetically ordered state and that with h^i = 
corresponding to the PM state are obtained, and the en- 
ergies per site E = Q.-\-iin are compared for the AF, AF2, 
AFC, and PM states to determine the ground state. The 
density of state per site 

/j2 7 1 ^'-^ 1 

(Zi:r rir ^ — ' IT 
^ ' fT,a— 1 

is also calculated to examine the gap, where n^ is the 
number of the sites in the unit cell in the sense of the sub- 
lattice formalism with the magnetic orderings (jic = 4 for 
the AF2 state, Uc = '2 for the AF and AFC states, and 
Tic = 1 for the PM state), and the k integration is over the 
corresponding Brillouin zone (see Fig. [T](b)). In Eq. ^, 
77 —> limit is evaluated using the standard extrapolation 
method by calculating D{lu) for 77 = 0.1, 0.05, and 0.025. 
The numerical error after this extrapolation is estimated 
to be of order 10"'^, so the gap is identified as the region of 
Lo around w ~ where the extrapolated D{lu) is less than 
10~^. We also compute the magnetic order parameter per 
site 

I "- 

M = — Vsign(a)((7iat) - ("-ai)) 
nc ^ 

a— 1 

and the double occupancy per site 



a=l 



dU 



where {Uacr) and {na-\na\) are the expectation values of 
riacr and na^riai, respectively. 



IV. MAGNETISM AND MOTT TRANSITION 
A. Phase diagram 

Fig. [7] shows the phase diagram at half-filling and zero 
temperature obtained by VGA on the (a) 3x4, (b) 2x6, 
and (c) 12c cluster, where the circles, triangles, and 
squares represent the AF, AF2, and AFC states, respec- 
tively. The numerical error of the ground-state energy 
per site due to the error of the number density n is at 
most of the order 10~*, so when the ground-state energy 
of a magnetic state is degenerate with that of other states 
(e.g. PM state) within the accuracy of 10~^ we have 
used non-filled circles, triangles, and squares to indicate 
it. (Later we call a solution as energetically disfavored if 
its energy is higher than the ground-state energy by more 
than 10"'^.) The crosses are the Mott transition points 
obtained in each cluster assuming that no magnetic or- 
der is allowed (i.e. Hm = 0). As is mentioned earlier, the 
cluster-shape dependence of the results is a measure of 
the finite-size effects in our analysis. 

In the region U > W{= 8), the results of the three 
clusters agree rather well, and for t' < 0.65 the AF state 
is realized, while 0.8 < t' the AFC state is stable, and 
the PM state is realized between the AF and AFC states. 
There is no region where the incommensurate AF2 phase 
is realized ioT U >W, so our VGA result disagrees with 
that of Ref. [ll|. When U < W, the three clusters give 
qualitatively different results and the cluster-shape de- 
pendence is not negligible. In this area, the 3x4 clus- 
ter yields the results qualitatively very similar to the 
mean-field approximation in Fig. [SJ and shows that the 
AF2 state is realized in the region 0.7 ^ t' < 0.85 and 
4.5 ^ U < 6.5. Other incommensurate states may be 
also realized around this area. 

The Mott transition takes place for 4 < U < 6. Our 
critical interaction strength Umi is larger than the results 
of VGA on the clusters up to 8 sitesJ^ In general, as a 
cluster becomes larger the kinetic energies simulated on 
it increase as an average, and Umi increases. Our result 
is consistent with this observation. 



B. Magnetic properties 

Now we study in detail the magnetic properties and 
first consider the strong coupling region. In general VGA 
gives the exact results regardless of the cluster size and 
shape when inter-site hopping interactions are set to be 
zero (i.e. t = t' = in the present model). But it does 
not necessarily mean that VGA is a good approximation 
in the strong coupling limit { U ^ t, t') even for small 
clusters. In fact, in the present model, the phase struc- 
ture is not determined by the ratio t/U or t' /U , but is 
determined by the ratio t/t' (or, Ji/ J2 in the Heisenberg 
model) in the strong coupling region. Therefore the de- 
tailed analysis of this region is not a trivial check of the 
calculations. The order parameter of the AF and AFC 
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FIG. 8: (Color online) The order parameter of the AF state at (a) U — 30 and (b) U = 15 and that of the AFC state at (c) 
[/ = 30 and (d) [/ = 15 as a function oft' obtained by VGA using the 3x4 (circles), 2x6 (triangles), and 12C (crosses) clusters. 
The critical couplings i^i separating the AF and PM phases and fc2 separating the AFC and PM phases are also shown by 
the vertical full (3x4), dotted (2x6), and dash-dotted (12C) lines. In (c) and (d) the data points for t' < 4^2 are energetically 
disfavored since their energies are higher than the PM solutions by more than 10~^. 



states is computed as a function of t' at f7 = 30 and 
[7 = 15 in Fig. mby VGA on the 3x4 (circles), 2x6 (tri- 
angles), and 12C (crosses) clusters. The critical couplings 
t'^i separating the AF and PM phases and t'^2 separating 
the AFC and PM phases are also shown by the vertical 
full (3x4), dotted (2x6), and dash-dotted (12C) lines. 
In Fig. |8] (c) and (d) the data points for t' < t'^2 corre- 
spond to energetically disfavored solutions because their 
energies are higher than the PM solutions by more than 
10-3. 

As is seen in Fig. [SJ the cluster-shape dependence is 
rather small, especially at C/ = 30. The transition be- 
tween AF and PM states is of the second order since 
the order parameter smoothly goes to zero and there are 
no energetically disfavored AF solutions outside the AF 
phase. The transition between the AFC and PM states is 
first order (level crossing), since energetically disfavored 
AFC solutions exist outside the AFC phase. These na- 
ture of the phase transitions agree with the analysis of 
the Heisenberg model^"— and VCA on 2x2 cluster J^iii 

The window of the PM phase between the AF and 
AFC phases is t'^^ = 0.63 - 0.67 <t' < t'^^ = 0.80 - 0.82 



at f/ = 30 in our analysis. In the analysis of the 
Heisenberg model, the PM region appears in the range 
0.4 < J2/J1 < 0.6, which translates into the region 
0.63 "^ t' < 0.78 in the Hubbard model in the strong 
coupling limit and agrees reasonably well with our re- 
sults. Among the three clusters, the window of the 2x6 
cluster is closest to the result of the Heisenberg model. 
This may be related to the proposal of Ref. |4^ in the 
Heisenberg model that an array of spin singlets sponta- 
neously formed on 2x2 plaquettcs is realized in the PM 
(magnetically disordered) ground state and only the 2x6 
cluster exactly contains three units of such 2x2 plaque- 
ttes. 

Our window of the PM state is much wider than the 
VCA on the 8-site clusterji^ mainly because our t'^^ is 
much smaller than there. Since it is reported that spin 
structures extended to a few sites such as the dimer 
states^"— and spin singlets on 2 x 2 plaquettes^ will be 
realized in this PM region, it seems to be natural to ex- 
pect that the PM region becomes wider for larger clus- 
ters, as is observed here. In addition to this discrepancy 
in the large U limit, in Ref. [ij the width of the PM state 
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FIG. 9: (Color online) The order parameter of the (a) AF 
state at t' = 0.6 and (b) AFC state at t' — 0.9 as a function of 
U obtained by VCA using the 3x4 (circles), 2x6 (triangles), 
and 12C (crosses) clusters. The critical interaction strength 
Uc separating the magnetic and PM phases phases is also 
shown by the vertical full (3x4), dotted (2x6), and dash- 
dotted (12C) lines. 



dotted (12C) lines. 

The behavior of the AF order parameter is similar 
down to [/ ~ 5 for the three clusters, however in the 
case of the 12C cluster the AF solutions below U < 5.7 
are energetically disfavored compared to the PM solu- 
tions so a first order transition takes place at Uc = 5.7. 
The other two clusters predict that the transition is of the 
second order since there are no energetically disfavored 
AF solutions below Uc- 

In the case of the AFC solutions, we found that there 
arc typically two solutions, one with large M and the 
other with smaller M. The energies of the two solutions 
cross around C/ ~ 6 ~ 8, which results in a discontinuous 
change of the order parameter in Fig.[5](b). The disconti- 
nuity of the AFC order parameter takes place (though at 
much smaller U) also in the mean-field approximation!^ 
All the three clusters predict that at t' = 0.9 the tran- 
sition between the AFC and PM states is of the second 
order since there are no energetically disfavored AFC so- 
lutions below Uc- 

Next we study in detail the spectral function by VCA 
using the 3x4 cluster. This cluster yields results qual- 
itatively similar to the mean-field approximation in the 
region U < W and indicates the interesting possibility 
that the AF2 state is realized. It seems to us, though 
maybe subjective to some extent, that among the three 
clusters used here, the 3x4 cluster simulates best the cor- 
relations in the bulk, and the fact that the stability of the 
AF2 phase is also obtained in the mean-field approxima- 
tion suggests that it may not be due to some artifacts 
intrinsic to the 3x4 cluster but is worth being further 
studied as an interesting possibility. 



decreases as U increases in the region U > W, while in 
our result the width increases toward the large U limit 
as U increases. 

The fact that our t[i and t'^2 i^ the large U limit are 
close to the results of the Heisenberg model indicates that 
the correlations within and between composite structures 
of spins such as dimers and spin singlets on 2 x 2 plaque- 
ttes are simulated well on the 12-site clusters for U > W, 
which in turn supports the validity of our results about 
the absence of the AF2 state in that region since the AF2 
ordering is also like a composite structure in the sense 
that its unit cell is four sites, and the analysis of the AF2 
ordering requires simulating the correlations within and 
between these unit cells. 

Next we consider the region U < W. Fig. ^ shows 
the order parameter of the (a) AF state at t' = 0.6 and 
(b) AFC state at t' = 0.9 as a function of U obtained 
by VCA using the 3x4 (circles), 2x6 (triangles), and 
12C (crosses) clusters. The critical interaction strength 
Uc separating the magnetic and PM phases phases is also 
shown by the vertical full (3x4), dotted (2x6), and dash- 



In Fig. [TU] we show the spectral function for the AF 
{f = 0.6), AF2 (i' = 0.75), and AFC {f = 0.9) solutions 
a,tU — 6 calculated by VCA using the 3x4 cluster, where 
the magnetic Brillouin zones are depicted in Fig. [T] (b) 
and the x and y directions arc indicated in Fig. [51 The 
mean-field dispersions are also plotted (full lines). The 
order parameters computed by VCA are M = 0.60 for the 
AF, M = 0.26 for the AF2, and M = 0.29 for the AFC 
solutions, while the mean-field prediction is M = 0.83 for 
the AF and AF2, and AFC states. (The coincidence of 
the value M for the three magnetic states is purely acci- 
dental.) Though the quasiparticle dispersions in VCA are 
similar to the mean-field dispersions far from the Fermi 
level, they are largely modified near the Fermi level due 
to the electron correlations. The asymmetry of the spec- 
tral functions in the x and y directions is observed in the 
AF2 and AFC solutions. As is seen in Fig. [1] (b), for 
example, F — > Ky part is identical to M — > Kj, part for 
the AF state, it is identical to (tt, — 7r/2) -^ (tt, tt/2) part 
for the AF2 state, and it is identical to K^ -^ M part for 
the AFC state. 




(c) f=Q.9 U=6 




(AFC phase) 



FIG. 10: (Color online) The spectral function of the (a) 
AF (t' = 0.6), (b) AF2 (i' = 0.75), and (c) AFC (i' = 0.9) 
solutions at U — 6 calculated by VGA using the 3x4 clus- 
ter. The Lorentzian broadening with rj = O.lt is used in all 
cases. The mean-field dispersions are also plotted (full lines). 



C. Mott transition 

Next we study the Mott transition in detail. Fig. [TT] 
shows (a) the Mott gap A and (b) double occupancy Docc 
as functions of U a.tt' = 0.8 obtained by VGA on the 3x4 
(circles), 2x6 (triangles), and 12C (crosses) clusters, as- 
suming that no magnetic order is allowed (i.e. /im = 0). 
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FIG. 11: (Color online) (a) The Mott gap A and (b) 
double occupancy Docc as functions of U at t' — 0.8 ob- 
tained by VGA on the 3x4 (circles), 2x6 (triangles), and 12G 
(crosses) clusters, assuming that no magnetic order is allowed. 
In (b) the Mott transition points are indicated by the arrows. 



The Mott gap closes continuously a.t U ~ Umi and below 
Umi there are no energetically disfavored Mott insulator 
solutions, so the Mott transition is of the second order. 
In general, Mott transitions are predicted to be first order 
in the variational cluster approach with bath degrees of 
freedom where hybridization between the bath sites and 
cluster sites is treated as a variational parameter42i^ In 
these analyses, the coexisting metal and insulator solu- 
tions, leading to the first order transition, differ by the 
value of these hybridization parameters. Our analysis 
does not have bath degrees of freedom and technically 
this will be the origin of the difference. It remains to be 
clarified which is the correct picture. 

Next we study in detail the spectral densities and re- 
lated quantities. The spectral density is defined by 



A(ka,uj) — hiiG{k<7,Lu + IT]) 

TT 

where the Green function G{ka, z) is expressed as 

1 



G{ka, z) 



z ~ Sk - T.{ka, z) 



(10) 
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FIG. 12: (Color online) The spectral function A{k,u}) of the paramagnetic state for (a) U — 12 (Mott insulator), (b) U — 8 
(Mott insulator), and (c) C/ = 4 (metal) at t' — 0.75 calculated by VGA using the 3x4 cluster. The spectral function A^{k,uj) 
of the self-energy is also calculated for the same U and t' ((d), (e), and (f)). The Lorentzian broadening with rj = O.li is used 
in all cases. The full lines are the noninteracting band structure at the half-filling. 



in terms of the self-energy T,{ka, z), whose spectral rep- 
resentation is^ 



S(fcCT, z) 



9ka 






afe.(O>0. (12) 



Useful information on gk,y and auaiO can be obtained^ 
by comparing the jzj — >■ oo expansions of the Eq. (jlip 
with p^ and that of the expression 



G{kn,z) = {n\{cka 



1 



z~H + Eo 



1 



-"fccr . 



-Cka}\n), (13) 



z + H-Eo 
where \fl) and Eq are the ground state and ground-state 



energy of H, and for the Hamiltonian ([T]), 

/•oo 



where (no-) is the average number per site of electrons 
with spin a in the ground state \Q). 

Here we consider the paramagnetic state at half-filling 
and set {n±^) = 1/2. When the spectral weight (TkcriO 
becomes dominated by a single pole of the dispersion S.k , 
the Green function is given by 



G{k<7,z) 



z - Sk- 



CP/4 ' 

z-(.k 



(14) 



where it = £k + gka- When U is large, gka ~ U/2 is 
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cancelled by the chemical potential /i ~ t//2 in Sk at 
half-filling, so ik becomes more or less the same as the 
dispersion of the metallic state. Also, the atomic limit 
t = t' = yields the self-energy of the form in Eq. p^ 
with ^k = 0, which suggests that ^k will be almost inde- 
pendent of U for large U. Therefore, when U is large so 
that U ^ l^fcl, l^fel, the poles of the Green function P^ 
are given by 



UJ± = -{Ek + ^k) ± 



2' 



(15) 



thus the original band ik splits into the upper and lower 
Hubbard bands (of almost equal weights) and a gap of 
width U opens. As is observed here, the spectral density 
of the self-energy I](fccr, z), 



A^{ka,uj) = lmS{k<T,uj + ir]) 



(16) 



is a key to understand the Mott transition, which 
we study in detail together with the spectral density 
A{ka,uj). 

In Fig. [12] we show the spectral function A{k, to) of 
the paramagnetic state for (a) U = 12 (Mott insulator), 
(h) U ^ 8 (Mott insulator), and (c) [/ = 4 (metal) at 
i' = 0.75 calculated by VGA using the 3x4 cluster. The 
spectral function A^{k,u!) of the self-energy is also cal- 
culated for the same U and t' ((d), (e), and (f)). The 
nonintcracting band structure is plotted with full lines. 
Observing e.g. the region around Kj, the band shrinks 
toward the Fermi level compared to the nonintcracting 
case in the metallic phase (c), and it splits into upper and 
lower Hubbard bands in the Mott insulator state (b) and 
the gap opens. Below the Mott transition point ([/ = 4) 
the As{k,uj) seems to be almost featureless, but it be- 
comes dominated by the single pole of well-defined defi- 
nite dispersion above the Mott transition point (f/ = 8) 
and, comparing (d) and (e) the dispersion is almost in- 
dependent of U, as it is expected. 

Supplementing these analyses, we show in Fig. 1131 the 
momentum distribution function Uk for U = (crosses) 
, t/ = 4 (circles), and U = 8 (triangles) at t' = 0.75 
calculated by VGA using the 3x4 cluster. The sharp gap 
of Uk at the Fermi surface in the metal disappears in the 
Mott insulator phase. 



0.65, while 0.8 < t' the AFC state is stable, and the 
PM state is realized between the AF and AFC states. 
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FIG. 13: (Color online) The momentum distribution function 
Uk for [/ = (crosses), U — 4 (circles), and U — 8 (triangles) 
at t' = 0.75 calculated by VGA using the 3x4 cluster. 



There is no region where the incommensurate AF2 phase, 
suggested in the previous study,— is stable for U >W. 

In the strong coupling limit, our results quantitatively 
agree well with the analysis of the Heisenberg model, 
which supports the validity of our analysis in the region 
U>W. 

In the region U < W, the VGA on the 3x4 cluster 
yields the results qualitatively very similar to the mean- 
field analysis, and shows that the AF2 state is realized 
for 0.7 < t'/t < 0.85 and 4.5 < f/ < 6.5, though the 
cluster-shape dependence of the results are not negligible. 
So incommensurate states may be realized, but in the 
limited area around this region. 

The Mott transition takes place slightly below the 
band width W and is of the second order. The spectral 
density of the self-energy is featureless below the Mott 
transition point (metallic phase), but in the Mott insu- 
lator phase, it becomes dominated by single pole of a 
definite dispersion, which yields the Mott gap. A de- 
tailed study on the nature of the PM state between the 
AF and AFC states, such as the dimer phase, is an inter- 
esting subject, but it is unfortunately beyond the scope 
of VGA. 



V. SUMMARY AND CONCLUSIONS 

We have studied the magnetic properties and Mott 
transition in the Hubbard model on the square lattice 
by VGA and analyzed the phase diagram at half-filling 
and zero temperature. 12-site clusters of three different 
shapes (3x4, 2x6, and 12G cluster) are used to see the 
finite-size effects of the analysis. 

In the region U > W{= 8), the predictions of the three 
clusters agree well, and the AF state is realized for t' < 
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